On The Value of Leave-One-Out Cross-Validation Bounds
نویسنده
چکیده
A long-standing problem in classification is the determination of the regularization parameter. Nearly every classification algorithm uses a parameter (or set of parameters) to control classifier complexity. Crossvalidation on the training set is usually done to determine the regularization parameter(s). [1] proved a leave-one-out cross-validation (LOOCV) bound for a class of kernel classifiers. [2] extended the bound to Regularized Least Squares Classification (RLSC). We provide the (trivial) extension to multiclass. Our contribution is empirical work. We evaluate the bound’s usefulness as a selector for the regularization parameter for RLSC. We find that it works extremely poorly on the data set we experimented with (20 Newsgroups); the LOOCV bound consistently selects a regularization parameter that is too large.
منابع مشابه
QSRR Study of Organic Dyes by Multiple Linear Regression Method Based on Genetic Algorithm (GA–MLR
Quantitative structure-retention relationships (QSRRs) are used to correlate paper chromatographic retention factors of disperse dyes with theoretical molecular descriptors. A data set of 23 compounds with known RF values was used. The genetic algorithm-multiple linear regression analysis (GA-MLR) with three selected theoretical descriptors was obtained. The stability and predictability of the ...
متن کاملRadius-margin Bound on the Leave-one- out Error of the Llw-m-svm
To set the values of the hyperparameters of a support vector machine (SVM), one can use cross-validation. Its leave-one-out variant produces an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirement. To overcome this difficulty, several upper bounds on the leave-one-out error of the pattern recognition SVM have been derived. The most pop...
متن کاملNew upper bounds on cross-validation for the k-Nearest Neighbor classification rule
The present work addresses binary classification by use of the k-nearest neighbors (kNN) classifier. Among several assets, it belongs to intuitive majority vote classification rules and also adapts to spatial inhomogeneity, which is particularly relevant in high dimensional settings where no a priori partitioning of the space seems realistic. However the performance of the kNN classifier crucia...
متن کاملRadius-Margin Bound on the Leave-One-Out Error of a M-SVM
Using a support vector machine (SVM) requires to set the values of two types of hyperparameters: the soft margin parameter C and the parameters of the kernel. To perform this model selection task, the method of choice is cross-validation. Its leave-one-out variant is known to produce an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirem...
متن کاملModel Selection for the l2-SVM by Following the Regularization Path
For a support vector machine, model selection consists in selecting the kernel function, the values of its parameters, and the amount of regularization. To set the value of the regularization parameter, one can minimize an appropriate objective function over the regularization path. A priori, this requires the availability of two elements: the objective function and an algorithm computing the r...
متن کامل